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10 [0001] This application claims the benefits of co-pending U.S. Patent 

Provisional Application number 60/413,220 filed on September 23, 2002, and is 
incorporated herein by reference. 
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Background of the Invention 

1. Field of the Invention 

[0002] The present invention relates generally to the processing of video 
images, and more particularly to techniques for deinterlacing video images. 

2. Description of the Related Art 



[0003] MPEG-2 makes use of a chroma-to-luma sampling relationship known 
as 4:2:0, in which two color-difference signals are sampled at one half the rate of 

25 luma in both the vertical and horizontal directions. An MPEG-2 decoder typically 
produces an output signal in the 4:2:2 format in which the color signals are sampled 
at one half the luma rate in only the horizontal direction. The MPEG decoder must 
therefore upsample the 4:2:0 encoded data in the vertical direction to produce the 
desired 4:2:2 output. In addition, MPEG-2 video sequences may be encoded as 

30 either a progressive or interlaced image sequence. A problem occurs when an 

image actually encoded as progressive is decoded as interlaced for the purposes of 
the chroma upsampling. In this case, the chroma samples are incorrectly 
upconverted and positioned relative to the corresponding luma samples, resulting in 
artifacts in areas of vertical color transitions. 
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[0004] MPEG-2 compression is in common use in today's digital video 
systems. Since MPEG-2 is a digital video compression method, the video signal 
must be digitally sampled prior to being compressed. The sampled digital video 
signal is composed of three components: a luminance (aka, luma') signal which 
5 contains brightness information but no color, and two color or chrominance (aka, 
'chroma') signals which are a scaled version of the difference between red and 
luma and between blue and luma (aka, 'R-Y' or Vr\ and 'B-Y' or 'Cb\ 
respectively). Inherent in the MPEG-2 encoding is a relationship between the 
sampling rates and the relative spatial positioning of these luma and chroma 
10 samples. 

[0005] While the MPEG-2 standard supports a variety of signal formats and 
resolutions, the common type in use today for standard definition video signals uses 
a luma/chroma sampling relationship known as 4:2:0. This standard dictates that 

15 the two chroma signals are sampled at half the rate of the luma signal in both the 
vertical and horizontal directions - i.e., each of the two chroma signals has one 
quarter the bandwidth of the luma signal. A chroma sample is spatially positioned 
halfway between a pair of luma samples in the vertical direction, and may be 
either halfway between a pair of luma samples in the horizontal direction or may be 

20 coincident with one of the two luma samples. It is only the vertical relationship 
which is at issue here. 

10006] Even though MPEG-2 video data is in the 4:2:0 format, most MPEG-2 
decoders produce a digital video output which is in the 4:2:2 format. With 4:2:2, 

25 the chroma data is sampled at half the luma rate in only the horizontal direction. 
There is one of each type of chroma sample - Cr and Cb - for each luma sample in 
the vertical direction. Because of this difference in the input and output sampling 
rates for chroma, an MPEG-2 decoder must \ipconvert' or 'upsample' the chroma 
signals vertically to produce an output in the 4:2:2 format. Various techniques can 

30 be used for this upconversion process, and range from simple replication of one 
chroma sample over two luma samples, to linear interpolation, to more 
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sophisticated sample rate conversion using digital FIR filters. 

[0007] One other key aspect of MPEG-2 is that the compressed video can be 
in either progressive or interlaced format. Although the vast majority of MPEG-2 
• 5 decoders output video in the interlaced format, the original video signal could have 
been compressed in either of the progressive or interlaced formats, depending on 
the source's characteristics. Control flags within the MPEG-2 data stream indicate 
which format was used for each video frame. It is important to note that if a given 
video signal has been compressed in one format, it must also be decoded in that 
10 same format or there can be problems with the positioning of decoded luma and 
chroma samples relative to each other. 

[0008] Figure 1 illustrates what can happen when a video frame of the prior 
art encoded as progressive is treated as interlaced. The leftmost column 10 

15 indicates a single column of video samples (the horizontal dimension is ignored for 
the purposes of this discussion) which has been encoded as progressive. For each 
pair of luma samples 20, represented by the open circles and numbered from zero 
at the top of the column in increasing order towards the bottom of the column, there 
is a single chroma sample 30 (note that only one chroma sample 30 per luma pair 

20 20 is shown - there is actually a pair of chroma samples, Cr and Cb) located 

halfway between the luma pair 20. The chroma samples have been designated with 
letters, starting with 'A' and progressing vertically downwards. 

[0009] The second column 50 indicates the luma/chroma relationship for 
25 images encoded as interlaced. Note that the chroma samples for an interlaced 

image are not spatially located at equidistant from the luma samples as the chroma 
samples for a progressive image. The actual chroma samples from the progressive 
image are derived from an even/odd line pair, not an even/even or odd/odd line pair 
as for an interlaced image. Even worse, when the chroma samples are vertically 
30 upsampled to form the 4:2:2 output, half of the original chroma samples are 
missing from each of the two fields and the upconversion is performed twice on 
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only half the data. This results in a distortion of the image. This distortion is 
especially apparent when the interlaced output fields of the MPEG-2 decoder are 
combined back into the original progressive image by an interlaced-to-progressive 
video format converter. 

5 

[0010] The effects of this distortion are shown in the remaining columns in 
Figure 1 . The third column 60 indicates what takes place when simple sample 
replication is used to perform the vertical chroma upconversion. The left set of 
upconverted values 70 are the correct ones based on the original progressive 

10 encoding. The right set of upconverted values 80 are those derived from improperly 
interpreting the video frame as interlaced. Note that the ordering of the 'interlaced' 
chroma upconverted samples actually reverses direction every few samples, 
resulting in a periodic retrograde sequence. This reversal can result in visible 
artifacts in the region of vertical chroma transitions such as the border between two 

15 objects of different colors. These artifacts somewhat resemble interlace motion 
artifacts, which is reasonable since they stem from an interlaced scan ordering. 
The fourth column 90 indicates the same relationship, but assumes that linear 
interpolation has been used for the upconversion process instead of sample 
replication. Again, the ordering of the interlaced' upconverted chroma samples 

20 indicates the periodic retrograde sequence which results in visible artifacts. 

[0011] Accordingly, what is needed is a method and system that detects and 
repairs video image distortion caused by incorrect decoding of MPEG-2 images. 
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Summary of the Invention 

[0012] The present invention meets these needs by being utilized inside of an 
interlace-to-progressive scan converter, and identifies and repairs interlaced video 
* 5 input streams with this problem in the process of converting them to a 

progressively scanned output signal. The problem addressed by the present 
invention is that of video encoded as progressive being treated as interlaced for the 
purposes of the chroma upconversion from 4:2:0 to 4:2:2. It should be appreciated 
that the present invention and various discreet features thereof can be implemented 
10 in numerous ways, including as a process, an apparatus, a system, a device, or a 
method. Furthermore, various discrete apparatus of the invention can be 
implemented in software or hardware. Several inventive embodiments of the 
present invention are described below. 

15 [0013] In one embodiment of the present invention, a method for removing 
MPEG-2 chroma upconversion artifacts in a video stream is disclosed. The method 
includes detecting a presence of artifacts in an incorrectly upsampled MPEG-2 
video stream and removing the presence of artifacts resulting in an artifact free 
video stream. 

20 

[0014] In another embodiment of the present invention, a video system for 
producing a corrected progressive MPEG-2 output from an MPEG-2 compressed 
video source includes an MPEG-2 compressed video source, a progressive MPEG- 
2 decoder having an input coupled to an output of the an MPEG-2 compressed 
25 video source, a buffer having an input coupled to an output of the progressive 

MPEG-2 decoder, a video processing module having an input coupled to an output 
of the buffer, and a progressive display having an input coupled to an output of the 
video processing module. 

30 [0015] In yet another embodiment of the present invention, a video system for 
producing a corrected progressive MPEG-2 output from an MPEG-2 compressed 
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video source includes an interlaced MPEG-2 decoder with an input coupled to an 
output of the MPEG-2 compressed video source, a deinterlacer with an input 
coupled to an output of the interlaced MPEG-2 decoder, a buffer with an input 
coupled to an output of the deinterlacer, a video processing module with an input 
5 coupled to an output of the buffer, and a progressive display with an input coupled 
to an output of the video processing module. The video processing module contains 
the previously described invention for detection and repair of MPEG-2 chroma 
upconversion artifacts. 

10 [0016] In a final embodiment of the present invention, a video system for 
producing a corrected progressive MPEG-2 output from an MPEG-2 compressed 
video source includes an MPEG-2 compressed video source, an interlaced MPEG- 
2 decoder with an input coupled to an output of the MPEG-2 compressed video 
source , a buffer with an input coupled to an output of interlaced MPEG-2 decoder, 

15 a deinterlacer/video processing module with an input coupled to an output of the 
buffer, and a progressive display with an input coupled to an output of the video 
processing module. 
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Brief Description of the Drawings 

[0017] The present invention will be readily understood by the following 
detailed description in conjunction with the accompanying drawings wherein: 

5 

[0018] Figure 1 illustrates the MPEG-2 decoder chroma upsampling problem 
of the prior art. 

[0019] Figure 2 illustrates a way to remove or reduce the primary negative 
10 effects of interlaced chroma upconversion of progressive images, in accordance 
with the present invention. 

[0020] Figure 3 illustrates a method for detecting MPEG-2 decoder interlace 
chroma upsampling artifacts, in accordance with the present invention. 

15 

[0021] Figure 4 illustrates a video system in which an MPEG-2 compressed 
video source produces a corrected progressive MPEG-2 output, in accordance with 
an embodiment of the present invention. 

20 [0022] Figure 5 illustrates a video system in which an MPEG-2 compressed 
video source produces a corrected progressive MPEG-2 output, in accordance with 
an embodiment of the present invention. 

[0023] Figure 6 illustrates a video system in which an MPEG-2 compressed 
25 video source produces a corrected progressive MPEG-2 output, in accordance with 
an embodiment of the present invention. 
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Detailed Description of the Preferred Embodiments 
[0024] Figure 1 was previously described in relation to the prior art. 

" 5 [0025] Figure 2 illustrates a way to remove or reduce the primary negative 
effects of interlaced chroma upconversion of progressive images. The calculated 
chroma column 100 is the same as the calculated chroma column 60 from Figure 1 
for upconversion by simple replication of chroma samples. The third column 110 
of Figure 2 illustrates what happens when a simple vertical lowpass filter is used on 

10 the chroma data. The lowpass filter shown is calculated in the following manner as 
shown in equation 1, where 'S N ' is the chroma sample at line 'N': 

[0026] Lowpass filtered S N = (S^ + 2*S N + S N+1 )/4 Equation 1 

15 [0027] For the interlaced upconversion, the lowpass filtering operation 

removes the direction reversal which occurs in the unfiltered upconversion. While 
the simple lowpass filter described above can be used to reduce or correct this 
problem, it should also be noted that other types of lowpass filters could be used, 
including simple averaging through more sophisticated FIR filters. 

20 

Detection of the Problem 

[0028] Although the lowpass filter correction method described above could 
be applied to all decoded video signals, it is preferable to apply it only to video 
signals which actually have had incorrect upsampling performed. This requires 

25 that there be a means to detect when the faulty upconversion has been performed. 
Such a detection means is illustrated in Figure 3. The detection method is based on 
the vertical frequency detection technique described in the DVDO patent 
application titled "Method and Apparatus for Detecting Frequency in Digital Video 
Images", U.S. Patent application serial number 09/372,713, previously 

30 incorporated by reference. In particular, a partial DFT is performed on four 

vertically aligned chroma samples to detect the specific frequency associated with 
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interlace artifacts. In addition, the partial DFT is performed twice, once with an 
even numbered line as the reference point and once with an adjacent odd numbered 
line as the reference point. 

5 [0029] This is illustrated in figure 3 for three different cases 120, 150 and 170 
(indicated by the large boxed-in areas) which are characteristic of a moving vertical 
chroma transition. Such a transition can occur in either a video sequence with a 
moving vertical edge, or for horizontally adjacent pixels in a static image with a 
diagonal vertical chroma transition. The key to the detection method is that the 
10 even and odd referenced partial DFTs yield very different results depending on the 
alignment of the chroma transition relative to even and odd lines. 

[0030] This is shown in figure 3 for the three cases. In the first case 120, a 
sharp chroma transition 130 occurs between chroma samples E and F. In the next 

15 two cases, the chroma transition 140 moves up until it occurs between samples D 
and E in the third case 150, with the chroma transition 160 for the second case 170 
being halfway between the first 120 and third 150. Partial DFTs, also called 
frequency detection values, are computed for each line, and the absolute value of 
the frequency detection values are summed for all the even lines 180 and separately 

20 for all the odd lines 190. In all three cases there is a significant difference between 
the even 180 and odd line sums 190. As the vertical chroma transition moves 
upward in the three cases of Figure 3, it can be seen that the even 180 and odd line 
sums 190 change from the even being largest to the odd being largest. 
[0031] It is this difference between even and odd frequency detection values 

25 that forms the basis of the detection mechanism. For video sources with the 

chroma upconversion problem in the neighborhood of a vertical chroma transition, 
when the even line frequency detection value is high, the odd is typically low, and 
vice versa. Video sources which do not have the incorrectly upconverted chroma 
tend not to show this even/odd difference at all, or at least to a much, much lesser 

30 degree. This difference can be used to distinguish between frequency detection 
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values for sources with upconversion problems and those with properly 
upconverted chroma. 

[0032] The detection scheme of the present invention takes the absolute value 
5 of the difference of the values of the even and odd line referenced frequency 

detection values for each pixel location in the current video field. These difference 
magnitudes are then summed over the entire field. Since the even and odd 
frequency detection values are typically different for a problem source, a relatively 
large sum results. The same process is then performed on the same chroma video 
10 data, but with a vertical lowpass filter operation performed on the chroma data first. 
As described in the previous section, the lowpass filter removes the artifacts 
associated with the incorrect chroma upconversion. Since the even and odd 
frequency detection values for the lowpass filtered data tend to be very similar, the 
field sum is relatively low. 

15 

[0033] This detection scheme applied to a video signal without the chroma 
upconversion problem yields two frequency detection sums which are very similar 
to each other since neither the original video data nor the lowpass filtered data has 
the upconversion problem. However, the two frequency detection sums derived 

20 from a video signal which does have the chroma upconversion problem yields 

sums which are very different from each other. This difference is typically greater 
than 10 to 1, and provides a clear differentiating factor which can be used to 
identify video sources with this problem. Once a problem source has been 
identified, the chroma upconversion artifacts can be removed with a vertical 

25 lowpass filter. 

[0034] Figure 4 illustrates a video system 200 in which an MPEG-2 
compressed video source produces a corrected progressive MPEG-2 output, in 
accordance with an embodiment of the present invention. The system includes an 
30 MPEG-2 compressed video source 210, a progressive MPEG-2 decoder 220 having 
an input coupled to an output of the an MPEG-2 compressed video source 210, a 
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buffer 230 having an input coupled to an output of the progressive MPEG-2 
decoder 220, a video processing module 240 having an input coupled to an output 
of the buffer 240, and a progressive display 250 having an input coupled to an 
output of the video processing module 250. The video processing module 240 
5 contains the previously described invention for detection and repair of MPEG-2 
chroma upconversion artifacts. 

[0035] Figure 5 illustrates a video system 260 in which an MPEG-2 
compressed video source produces a corrected progressive MPEG-2 output, in 

10 accordance with an embodiment of the present invention. The system includes an 
MPEG-2 compressed video source 210, an interlaced MPEG-2 decoder 280 with 
an input coupled to an output of the MPEG-2 compressed video source 210, a 
deinterlacer 290 with an input coupled to an output of the interlaced MPEG-2 
decoder 280, a buffer 230 with an input coupled to an output of the deinterlacer 

15 290, a video processing module 240 with an input coupled to an output of the 
buffer 230, and a progressive display with an input coupled to an output of the 
video processing module 240. The video processing module 240 contains the 
previously described invention for detection and repair of MPEG-2 chroma 
upconversion artifacts. 

20 

[0036] Figure 6 illustrates a video system 330 in which an MPEG-2 
compressed video source produces a corrected progressive MPEG-2 output, in 
accordance with an embodiment of the present invention. The system includes an 
MPEG-2 compressed video source 210, an interlaced MPEG-2 decoder 280 with 

25 an input coupled to an output of the MPEG-2 compressed video source 210, a 

buffer 230 with an input coupled to an output of interlaced MPEG-2 decoder 280, a 
deinterlacer/video processing module 370 with an input coupled to an output of the 
buffer 230, and a progressive display 250 with an input coupled to an output of the 
video processing module 370. The video processing module 370 contains the 

30 previously described invention for detection and repair of MPEG-2 chroma 
upconversion artifacts and a deinterlacer. 
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,0037, This embodiment has several advantages over the system 260 as 
shown in Fig. 5. Sinee mneh of what is needed to implement the present invennon 
is available in a deinterlacer, the two modules ean be easily merged and results m a 
m ore eeonomieal system to manufacture. Additionally, one less frame penod of 
delay is realized in the system 330 sinee a separate implementation of the present 
invention incurs a single-frame period of delay in analyzing the v.deo data. 

[461 While this invention has been described in terms of several preferred 
embodfrnents, it will be appreciated that those skilled in the art, upon reading the 
preceding specifications and studying the drawings, will realize vanous alterations, 
additions, permutations, and equiva.ents as fall within the true spirit and scope of 
the invention. 



15 What is claimed is: 
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